Image processing device, image processing method, program and image processing system

ABSTRACT

It is desirable to provide a technology capable of further appropriately adjusting the luminance of the endoscopic image. Provided is an image processing device including: an area extraction unit configured to extract, as an extraction area, an area corresponding to the size of an insertion unit from an endoscopic image based on imaging by an image sensor; and an exposure control unit configured to perform exposure control on a basis of an output value of the image sensor in the extraction area.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation application of U.S. patentapplication Ser. No. 16/823,848, filed Mar. 19, 2020, which is acontinuation application of U.S. patent application Ser. No. 15/546,111,filed Jul. 25, 2017, now U.S. Pat. No. 10,623,651, which is a nationalstage entry of PCT/JP2015/082324 filed Nov. 17, 2015, which claimspriority from prior Japanese Priority Patent Application JP 2015-025133filed in the Japan Patent Office on Feb. 12, 2015, the entire content ofwhich is hereby incorporated by reference.

TECHNICAL FIELD

The present disclosure relates to an information processing device, aninformation processing method, a program and an image processing system.

BACKGROUND ART

In recent years, image processing devices for processing an endoscopicimage based on imaging by an image sensor have gained in popularity(e.g., see Patent Literature 1). Meanwhile, a phenomenon may occur inwhich the endoscopic image is partially darkened by light shieldingcaused by, for example, the hood of the lens for transmitting light tothe image sensor. Hereinafter, such an area darkened by light shieldingin the endoscopic image is also simply referred to as “black area”.Moreover, a phenomenon in which such a black area occurs in theendoscopic image is also called “vignetting”.

CITATION LIST Patent Literature

-   Patent Literature 1:-   JP 2013-42998A

DISCLOSURE OF INVENTION Technical Problem

Here, due to the occurrence of the black area in the endoscopic image,there is a case where exposure control may be performed so that theluminance of the endoscopic image becomes excessively high. Thus, anarea other than the black area (hereinafter, also referred to as“observation area”) may become excessively bright. Accordingly, it isdesirable to provide a technology capable of further appropriatelyadjusting the luminance of the endoscopic image.

Solution to Problem

According to the present disclosure, there is provided an imageprocessing device including: an area extraction unit configured toextract, as an extraction area, an area corresponding to the size of aninsertion unit from an endoscopic image based on imaging by an imagesensor; and an exposure control unit configured to perform exposurecontrol on a basis of an output value of the image sensor in theextraction area.

According to the present disclosure, there is provided an imageprocessing device including: extracting, as an extraction area, an areacorresponding to the size of an insertion unit from an endoscopic imagebased on imaging by an image sensor; and performing exposure control bya processor on a basis of an output value of the image sensor in theextraction area.

According to the present disclosure, there is provided a program forcausing a computer to function as an image processing device including:an area extraction unit configured to extract, as an extraction area, anarea corresponding to the size of an insertion unit from an endoscopicimage based on imaging by an image sensor; and an exposure control unitconfigured to perform exposure control on a basis of an output value ofthe image sensor in the extraction area.

According to the present disclosure, there is provided an imageprocessing system including: a light source unit configured to emitlight; an image sensor configured to capture an endoscopic image byreceiving reflected light of the light emitted by the light source unit;and an image processing device including an area extraction unitconfigured to extract, as an extraction area, an area corresponding tothe size of an insertion unit from the endoscopic image, and an exposurecontrol unit configured to perform exposure control on a basis of anoutput value of the image sensor in the extraction area.

Advantageous Effects of Invention

As described above, according to the present disclosure, a technologycapable of further appropriately adjusting luminance of an endoscopicimage is provided. Note that the effects described above are notnecessarily limitative. With or in the place of the above effects, theremay be achieved any one of the effects described in this specificationor other effects that may be grasped from this specification.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram showing an exemplary configuration of an imageprocessing system according to an embodiment of the present disclosure.

FIG. 2 is an explanatory graph of the specific example of exposurecontrol.

FIG. 3 is an explanatory graph of the specific example of exposurecontrol.

FIG. 4 is a block diagram showing an exemplary detailed functionalconfiguration of an automatic exposure control unit.

FIG. 5 is a diagram showing an exemplary endoscopic image.

FIG. 6 is a diagram showing another exemplary peripheral area.

FIG. 7 is an explanatory diagram of a calculation example of a scopediameter in a case that two directions are used as the scanningdirection.

FIG. 8 is an explanatory diagram of a calculation example of the scopediameter in the case that two directions are used as the scanningdirection.

FIG. 9 is a diagram showing an exemplary black area frame map.

FIG. 10 is a flowchart showing an exemplary operation of exposurecontrol depending on the scope diameter.

MODE(S) FOR CARRYING OUT THE INVENTION

Hereinafter, (a) preferred embodiment(s) of the present disclosure willbe described in detail with reference to the appended drawings. In thisspecification and the appended drawings, structural elements that havesubstantially the same function and structure are denoted with the samereference numerals, and repeated explanation of these structuralelements is omitted.

Note that, in this description and the drawings, structural elementsthat have substantially the same function and structure are sometimesdistinguished from each other using different alphabets after the samereference sign. However, when there is no need in particular todistinguish structural elements that have substantially the samefunction and structure, the same reference sign alone is attached.

Note that description will be given in the following order.1. Embodiment of the present disclosure1.1. Exemplary system configuration1.2. Exemplary function configuration1.3. Functional detail of automatic exposure control unit

2. Conclusion 1. EMBODIMENT OF THE PRESENT DISCLOSURE [1.1. ExemplarySystem Configuration]

First, an exemplary configuration of an image processing systemaccording to an embodiment of the present disclosure will be describedwith reference to the drawings. FIG. 1 is a diagram showing an exemplaryconfiguration of an image processing system according to an embodimentof the present disclosure. As shown in FIG. 1, the image processingsystem 1 includes an image processing device 100, an insertion unit 200,a light source unit 300, a display unit 400, and an operation unit 500.

The light source unit 300 includes a white light source 310 and acondenser lens 320. The white light source 310 emits white light. Notethat this specification mainly describes examples of using white light,but the color of light is not limited in particular. Accordingly,instead of the white light source 310, light sources which emit visiblelight other than white may be used (e.g., instead of the white lightsource 310, RGB lasers which can perform variable control of RGB outputmay be used). The condenser lens 320 focuses the light emitted by thewhite light source 310 to a light guide 210 described below.

The insertion unit 200 can correspond to a scope to be inserted into abody. Specifically, the insertion unit 200 may be a rigid endoscope or asoft endoscope. The insertion unit 200 includes the light guide 210, anillumination lens 220, an imaging unit 230, and a memory 240. Theimaging unit 230 includes an objective lens 231, an image sensor(imaging element) 232, and an A/D (analog/digital) conversion unit 233.

The light guide 210 guides the light focused by the light source unit300 to the end of the insertion unit 200. The illumination lens 220diffuses the light that has been guided to the end by the light guide210, and irradiates an observation target (subject Su) with the diffusedlight. The objective lens 231 focuses the reflected light returning fromthe observation target (subject Su) to form an image on the image sensor232. The image sensor 232 outputs analog signals (endoscopic image)captured by receiving the reflected light to the A/D conversion unit233.

Note that the image sensor 232 has, for example, a primary color Bayerarray. In such a case, the endoscopic image obtained by the image sensor232 is a primary color Bayer image. The primary color Bayer image is animage in which each pixel has any of R, G, and B signals, and the RGBpixels are arranged in a staggered pattern. However, the image sensor232 is not limited to the primary color Bayer array. Namely, theendoscopic image is not limited to the primary color Bayer image. Forexample, the endoscopic image may be an image acquired by an endoscopeimaging method e.g., complementary-color method or frame-sequentialimaging method other than the primary color Bayer.

The A/D conversion unit 233 converts, on the basis of a control signaloutput from a control unit 130 described below, analog signals(endoscopic image) output from the image sensor 232 into digitalsignals, and outputs the digital signals (endoscopic image) to the imageprocessing device 100. The memory 240 stores a program for implementingfunction of the image processing device 100 when being executed by anoperation device (not shown).

Note that in the following description, the insertion unit 200 may bereferred to as “scope” as appropriate. A different scope can be used forendoscopic diagnosis depending on a diagnosis region. An identificationnumber for specifying a target diagnosis region and a function, such asa zoom function, is assigned to each scope, and in this specification,the identification number may be referred to as “scope ID”. The memory240 stores the scope ID.

The image processing device 100 includes an automatic exposure controlunit 110, an image processing unit 120, and the control unit 130. Theendoscopic image acquired by the imaging unit 230 is output to theautomatic exposure control unit 110 and the image processing unit 120.The automatic exposure control unit 110 is connected to the white lightsource 310 and the image sensor 232, and controls the white light source310 and the image sensor 232. The image processing unit 120 is connectedto the display unit 400. The control unit 130 is bidirectionallyconnected to the imaging unit 230, the image processing unit 120, thedisplay unit 400, and the operation unit 500, and controls thesecomponents.

The automatic exposure control unit 110 automatically performs exposurecontrol of the image sensor 232 such that the luminance of theendoscopic image acquired by the imaging unit 230 is a value appropriatefor observation (hereinafter, referred to as “appropriate value”). Theautomatic exposure control unit 110 will be described in detail below.The image processing unit 120 performs image processing on theendoscopic image captured by the imaging unit 230. The image processingunit 120 performs, for example, a tone transformation process and anoise reduction process. The image processing unit 120 outputs the imagesubjected to the image processing to the display unit 400.

The control unit 130 is connected to the imaging unit 230, the imageprocessing unit 120, the display unit 400, and the operation unit 500,and outputs control signals for controlling these. The display unit 400outputs the endoscopic image output by the image processing unit 120 toan image display device such as an endoscope monitor. The operation unit500 is an interface for receiving operations from a user. For example,the operation unit 500 includes a power switch for turning ON/OFF thepower supply, a shutter button for starting an imaging operation, a modeswitch button for switching an imaging mode and other various modes, andthe like.

The exemplary configuration of the image processing system 1 accordingto the embodiment of the present disclosure has been described above.

[1.2. Example of Exposure Control]

Subsequently, specific examples of exposure control by the automaticexposure control unit 110 will be described. FIGS. 2 and 3 areexplanatory graphs of specific examples of the exposure control. Asdescribed above, analog signals captured by the image sensor 232 areconverted to digital signals (endoscopic image) by the A/D conversionunit 233. In FIGS. 2 and 3, the output value from the image sensor 232is shown on the vertical axis. Moreover, the image plane illuminance ofthe image sensor 232 corresponding to each output value is shown on thehorizontal axis. Note that the output value from the image sensor 232may be a mean value of output values corresponding to each pixel.

Moreover, with reference to FIG. 2, an appropriate value of the outputvalue from the image sensor 232 is shown as “U0”, and the image planeilluminance of the image sensor 232 corresponding to the appropriatevalue U0 is shown as “L0”. As shown in FIG. 2, for example, it isassumed that the output value U1 from the image sensor 232 is largerthan the appropriate value U0. In such a case, the automatic exposurecontrol unit 110 performs exposure control so as to decrease the outputvalue from the image sensor 232 by dU1 (U1−U0=dU1).

On the other hand, with reference to FIG. 3, as in FIG. 2, theappropriate value of the output value from the image sensor 232 is shownas “U0”, and the image plane illuminance of the image sensor 232corresponding to the appropriate value U0 is shown as “L0”. As shown inFIG. 3, for example, it is assumed that the output value U2 from theimage sensor 232 is smaller than the appropriate value U0. In such acase, the automatic exposure control unit 110 performs exposure controlso as to increase the output value from the image sensor 232 by dU2(U0−U2=dU2).

For example, the exposure control may be performed by adjustingparameters for controlling exposure. A variety of parameters are assumedas the parameter for controlling exposure. For example, the parameterfor controlling exposure may include at least any one of an electronicshutter speed of the image sensor 232 and a gain by which the analogsignals captured by the image sensor 232 are multiplied. Alternatively,the parameter for controlling exposure may include brightness of thewhite light source 310 (alternatively, when an RGB laser is used insteadof the white light source 310, the exposure control may be performedthrough light source control by modifying outputs of respective RGB).

For example, the exposure control to decrease the output value from theimage sensor 232 by dU1 as shown in FIG. 2 may be executed by increasingthe electronic shutter speed by an amount corresponding to dU1, or maybe executed by decreasing a gain by which the analog signals captured bythe image sensor 232 are multiplied by an amount corresponding to dU1.Alternatively, the exposure control to decrease the output value fromthe image sensor 232 may be executed by weakening the brightness of thewhite light source 310 by an amount corresponding to dU1.

On the other hand, the exposure control to increase the output valuefrom the image sensor 232 by dU2 as shown in FIG. 3 may be executed bydecreasing the electronic shutter speed by an amount corresponding todU2, or may be executed by increasing a gain by which the analog signalscaptured by the image sensor 232 are multiplied by an amountcorresponding to dU2. Alternatively, the exposure control to increasethe output value from the image sensor 232 may be executed by increasingthe brightness of the white light source 310 by an amount correspondingto dU2.

The specific examples of the exposure control by the automatic exposurecontrol unit 110 have been described above.

[1.3. Functional Detail of Automatic Exposure Control Unit]

Subsequently, detailed function of the automatic exposure control unit110 will be described. FIG. 4 is a block diagram showing an exemplarydetailed functional configuration of the automatic exposure control unit110. As shown in FIG. 4, the automatic exposure control unit 110includes a size acquisition unit 111, an area extraction unit 112, andan exposure control unit 112. Hereinafter, each function of the sizeacquisition unit 111, the area extraction unit 112 and the exposurecontrol unit 113 will be described in detail. First, the sizeacquisition unit 111 acquires an endoscopic image from the imaging unit230.

FIG. 5 is a diagram showing an exemplary endoscopic image. As shown inFIG. 5, in an endoscopic image Im1, each pixel is arranged in a latticeshape. Here, as mentioned above, a phenomenon may occur in which theendoscopic image Im1 is partially darkened by light shielding caused by,for example, the hood of the lens for transmitting light to the imagesensor 232. Therefore, in the endoscopic image Im1, there is a blackarea Rb1 in addition to an observation area Rs1. Lines Hm indicateboundary lines between the black areas Rb1 and the observation area Rs1.The color density of each pixel represents the height of the luminanceof each pixel.

Here, there is a case where, due to the occurrence of the black area Rb1in the endoscopic image Im1, exposure control may be performed by theautomatic exposure control unit 110 so that the luminance of theendoscopic image Im1 becomes excessively high. Thus, the observationarea Rs1 may become excessively bright. Accordingly, a technology willbe described below which is capable of further appropriately adjustingthe luminance of the endoscopic image Im1 by reducing the possibilitythat the observation area Rs1 becomes excessively bright.

Specifically, the area extraction unit 112 extracts, as an extractionarea, an area corresponding to the size of the insertion unit 200 fromthe endoscopic image Im1 based on imaging by the image sensor 232. Then,the exposure control unit 113 performs exposure control on the basis ofan output value of the image sensor in the extraction area. That makesit possible to further appropriately adjust the luminance of theendoscopic image Im1 by reducing the possibility that the observationarea Rs1 becomes excessively bright.

For example, in the case of having determined that the shadow of theinsertion unit 200 is imaged in the endoscopic image Im1, the exposurecontrol unit 113 may perform exposure control based on an output valueof the image sensor in the extraction area. More specifically, in thecase of having determined that the shadow of the insertion unit 200 isimaged in the endoscopic image Im1, the exposure control unit 113 mayadjust parameters for controlling exposure on the basis of an outputvalue of the image sensor in the extraction area.

On the other hand, in the case of having determined that the shadow ofthe insertion unit 200 is not imaged in the endoscopic image Im1, theexposure control unit 113 may perform exposure control based on anoutput value of the image sensor in the entire endoscopic image Im1.Note that determination of whether or not the shadow of the insertionunit 200 is imaged in the endoscopic image Im1 may be made in any way.

For example, in the case that the luminance difference between thecenter area Ce and peripheral areas N1 to N4 is greater than a firstthreshold, the exposure control unit 113 may determine that the shadowof the insertion unit 200 is imaged in the endoscopic image Im1.Moreover, in the case that the luminance difference between the centerarea Ce and the peripheral areas N1 to N4 is less than the firstthreshold, the exposure control unit 113 may determine that the shadowof the insertion unit 200 is not imaged in the endoscopic image Im1. Inthe case that the luminance difference between the center area Ce andthe peripheral areas N1 to N4 is equal to the first threshold, theexposure control unit 113 may determine it as either case.

Alternatively, the exposure control unit 113 detects a first peak and asecond peak in sequence from the lower luminance side from thenumber-of-pixel distribution for each luminance of the endoscopic image,and then in the case that the luminance difference between the firstpeak and the second peak exceeds an upper limit value, the exposurecontrol unit 113 may determine that the shadow of the insertion unit 200is imaged in endoscopic image Im1. Moreover, in the case that theluminance difference between the first peak and the second peak does notexceed the upper limit value, the exposure control unit 113 maydetermine that the shadow of the insertion unit 200 is not imaged in theendoscopic image Im1.

In the example shown in FIG. 5, the luminance of the center area Ce isassumed to use the average luminance of the four pixels present in thecenter of the endoscopic image Im1, but the position of the center areaCe and the number of pixels are not limited. Similarly, the luminance ofthe peripheral areas N1 to N4 is assumed to use the average luminance ofthe four pixels in the four corners of the endoscopic image Im1, but theposition of the peripheral areas N1 to N4 and the number of pixels arenot limited. FIG. 6 is a diagram showing another example of theperipheral area. As shown in FIG. 6, instead of the peripheral areas N1to N4, the pixel columns present at the outermost sides of theendoscopic image Im1 may be used as peripheral areas N5 and N6.

Subsequently, the size acquisition unit 111 acquires the size of theinsertion unit 200. Note that in the following description, a scopediameter (diameter of the insertion unit 200) is used as the size of theinsertion unit 200, but instead of the scope diameter, other lengths ofthe insertion unit 200 (e.g., radius of the insertion unit 200 or thelike) may be used. The size acquisition unit 111 may acquire the scopediameter from a predetermined place. For example, the size acquisitionunit 111 may acquire the scope diameter (or, scope information includingthe scope diameter) by communicating with a scope body or externaldevices. Alternatively, the size acquisition unit 111 may acquire thescope diameter by calculation on the basis of the endoscopic image Im1.

The method of acquiring the scope diameter by calculation is not limitedin particular. For example, in the case of scanning from a first startposition of the endoscopic image Im1 toward a first target position in afirst scanning direction, the size acquisition unit 111 may calculatethe scope diameter on the basis of a first pixel position at which themagnitude relation between the luminance and the second threshold isfirst switched. At this time, the size acquisition unit 111 maycalculate twice the first distance between the first pixel position andthe center position as the scope diameter.

Note that the second threshold may be a fixed value, but may be set to avalue depending on a situation of the endoscopic image Im1. For example,the size acquisition unit 111 may set the second threshold on the basisof a number-of-pixel distribution for each luminance of the endoscopicimage Im1. More specifically, the size acquisition unit 111 may specifya median value of a number-of-pixel distribution for each luminance ofthe endoscopic image Im1 and set the median value as the secondthreshold.

Moreover, the scanning direction may be plural. The example where twodirections are used as the scanning direction will be described. FIGS. 7and 8 are explanatory diagrams of a calculation example of the scopediameter in the case where two directions are used as the scanningdirection. As shown in FIG. 7, in the case of scanning from the firststart position of the endoscopic image Im1 in the first scanningdirection Dr, the size acquisition unit 111 calculates a first pixelposition at which the magnitude relation between the luminance and thesecond threshold is first switched.

Moreover, as shown in FIG. 8, in the case of scanning from the secondstart position of the endoscopic image Im1 in the second scanningdirection DI, the size acquisition unit 111 calculates a second pixelposition at which the magnitude relation between the luminance and thesecond threshold is first switched. Then, the size acquisition unit 111calculates the scope diameter on the basis of the first pixel positionand the second pixel position.

More specifically, in the case that a difference between a firstdistance Wr between the first pixel position and the center position anda second distance Wl between the second pixel position and the centerposition is less than a third threshold (predetermined distance), thesize acquisition unit 111 may calculate a sum of the first distance Wrand the second distance Wl as the scope diameter. On the other hand, inthe case that the difference between the first distance Wr and thesecond distance Wl is greater than the third threshold, the sizeacquisition unit 111 may calculate twice the larger one of the firstdistance Wr and the second distance Wl as the scope diameter.

As shown in FIG. 7, the first start position may be a center position ofthe endoscopic image Im1, and the first target position may be an endposition of the endoscopic image Im1. Moreover, as shown in FIG. 8, thesecond start position may be also the center position of the endoscopicimage Im1, and the second target position may be an end position of theendoscopic image Im1. Since a subject Su is likely to appear with highluminance inside the endoscopic image Im1, in the case that theendoscopic image Im1 is scanned from the inside toward the outside, moreaccurate calculation can be done.

However, the first start position, the second start position, the firsttarget position and the second target position are not limited to theexamples shown in FIGS. 7 and 8. For example, the first start positionmay be the end position of the endoscopic image Im1, and the firsttarget position may be the center position of the endoscopic image Im1.Since the black area is likely to be present only at the end of theendoscopic image Im1, in the case that the endoscopic image Im1 isscanned from the outside to the inside, the calculation can be completedearlier.

Note that in the example mentioned above, the first scanning directionis the right direction and the second scanning direction is the leftdirection, but the first scanning direction and the second scanningdirection are not limited to such an example. For example, one of thefirst scanning direction and the second scanning direction may be anupward direction, and the other may be a downward direction. Moreover,one of the first scanning direction and the second scanning directionmay be an oblique direction (a direction from the upper right to thelower left, or a direction from the lower left to the upper right) andthe other may be an oblique direction (a direction from the upper leftto the lower right, or a direction from the lower right to the upperleft).

When the size acquisition unit 111 acquires the scope diameter, the areaextraction unit 112 extracts, as an extraction area, an areacorresponding to the scope diameter. For example, in the case that a mapthat defines the areas to be extracted (hereinafter, also referred to as“black area frame map”) is associated in advance with the scopediameter, the area extraction unit 112 acquires the black area frame mapcorresponding to the scope diameter acquired by the size acquisitionunit 111. FIG. 9 is a diagram showing an exemplary black area frame map.With reference to FIG. 9, in the black area frame map Mp, the areas tobe extracted are shown as “1” and the areas to be excluded are shown as“0”.

The area extraction unit 112 extracts pixels set as the area to beextracted in the black area frame map Mp from the endoscopic image Im1.FIGS. 7 and 8 show the extraction area Rw2 and the excluded area Rb2.The exposure control unit 113 then performs exposure control on thebasis of an output value of the image sensor in the extraction areaextracted by the area extraction unit 112. Note that the output valuefrom the image sensor 232 in the extraction area may be a mean value ofthe output values corresponding to each pixel of the extraction area. Itis thereby possible to further appropriately adjust the luminance of theendoscopic image Im1 by reducing a possibility that the observation areaRs1 becomes excessively bright.

Subsequently, an exemplary operation of the size acquisition unit 111,the area extraction unit 112 and the exposure control unit 113 asdescribed above will be described. FIG. 10 is a flowchart showing anexemplary operation of exposure control depending on the scope diameter.Note that the exemplary operation shown in FIG. 10 is a flowchartshowing an exemplary operation of exposure control depending on thescope diameter. Accordingly, the operation of exposure control dependingon the scope diameter is not limited to the exemplary operation shown inFIG. 10. Note that FIG. 10 shows the scope diameter as a black areaframe diameter.

As shown in FIG. 10, in the case that the luminance difference betweenthe center area Ce and the peripheral areas N1 to N4 (or N5, N6) is lessthan the first threshold in the endoscopic image Im1 (“No” in S11), thearea extraction unit 112 ends the operation. On the other hand, in thecase that the luminance difference between the center area Ce and theperipheral areas N1 to N4 (or N5, N6) is greater than the firstthreshold in the endoscopic image Im1 (“Yes” in S11), the areaextraction unit 112 binarizes each pixel of the endoscopic image Im1 bythe second threshold (S12).

Subsequently, the area extraction unit 112 performs line scan on thebinarized image (S13). Here, in the case that a difference between ascanning result in the right direction (hereinafter, also referred to as“right scanning result”) and a scanning result in the left direction(hereinafter, also referred to as “left scanning result”) is greaterthan the third threshold (“Yes” in S14), the area extraction unit 112sets the sum of the right scanning result and the left scanning resultto the black area frame diameter (S15), and the operation is shifted toS17.

On the other hand, in the case that the difference between the rightscanning result and the left scanning result is less than the thirdthreshold (“No” in S14), the area extraction unit 112 sets, to the blackarea frame diameter, twice the longer one of the scanning results (S16),and the operation is shifted to S17. Subsequently, the area extractionunit 112 selects the black area frame map Mp corresponding to the blackarea frame diameter (S17), and multiplies pixel values corresponding tothe black area frame map Mp and the endoscopic image Im1 (S18). By suchmultiplication, an extraction area Rw2 is extracted.

Subsequently, the exposure control unit 113 performs exposure control onthe basis of the multiplication result (S19). The exposure control unit113 performs exposure control on the basis of an output value of theimage sensor in the extraction area Rw2 extracted by the area extractionunit 112. It is thereby possible to further appropriately adjust theluminance of the endoscopic image Im1 by reducing a possibility that theobservation area Rs1 becomes excessively bright.

The detailed function of the automatic exposure control unit 110 hasbeen described above.

2. CONCLUSION

As described above, according to the embodiment of the presentdisclosure, the image processing device 100 is provided which includesthe area extraction unit 112 configured to extract an area correspondingto the size of the insertion unit 200 as an extraction area from theendoscopic image Im1 based on imaging by the image sensor 232 and theexposure control unit 113 configured to perform exposure control on thebasis of an output value of the image sensor in the extraction area.According to such a configuration, it is possible to furtherappropriately adjust the luminance of the endoscopic image Im1 byreducing a possibility that the observation area Rs1 becomes excessivelybright.

The preferred embodiment(s) of the present disclosure has/have beendescribed above with reference to the accompanying drawings, whilst thepresent disclosure is not limited to the above examples. A personskilled in the art may find various alterations and modifications withinthe scope of the appended claims, and it should be understood that theywill naturally come under the technical scope of the present disclosure.

Further, the effects described in this specification are merelyillustrative or exemplified effects, and are not limitative. That is,with or in the place of the above effects, the technology according tothe present disclosure may achieve other effects that are clear to thoseskilled in the art from the description of this specification.

Additionally, the present technology may also be configured as below.

(1)

An image processing device including:

an area extraction unit configured to extract, as an extraction area, anarea corresponding to the size of an insertion unit from an endoscopicimage based on imaging by an image sensor; and

an exposure control unit configured to perform exposure control on abasis of an output value of the image sensor in the extraction area.

(2)

The image processing device according to (1), including

a size acquisition unit configured to acquire the size of the insertionunit.

(3)

The image processing device according to (2),

in which the size acquisition unit acquires the size of the insertionunit from the insertion unit.

(4)

The image processing device according to (2),

in which the size acquisition unit acquires the size of the insertionunit by calculation on a basis of the endoscopic image.

(5)

The image processing device according to (4),

in which the size acquisition unit calculates the size of the insertionunit on a basis of a first pixel position at which a magnitude relationbetween a luminance and a threshold is first switched, in a case ofscanning from a first start position toward a first target position ofthe endoscopic image in a first scanning direction.

(6)

The image processing device according to (5),

in which the first start position is a center position of the endoscopicimage, and the first target position is an end position of theendoscopic image.

(7)

The image processing device according to (5),

in which the first start position is an end position of the endoscopicimage, and the first target position is a center position of theendoscopic image.

(8)

The image processing device according to (6) or (7),

in which the size acquisition unit calculates, as the size of theinsertion unit, a first distance between the first pixel position andthe center position or twice the first distance.

(9)

The image processing device according to any one of (5) to (8),

in which the first scanning direction is an upward direction, a downwarddirection, a left direction, a right direction or an oblique direction.

(10)

The image processing device according to (6),

in which the size acquisition unit calculates the size of the insertionunit on a basis of a second pixel position at which a magnitude relationbetween a luminance and a threshold is first switched and the firstpixel position in a case of scanning from a second start position of theendoscopic image in a second scanning direction.

(11)

The image processing device according to (10),

in which in a case that a difference between a first distance betweenthe first pixel position and the center position and a second distancebetween the second pixel position and the center position is less than apredetermined distance, the size acquisition unit calculates, as thesize of the insertion unit, a sum of the first distance and the seconddistance.

(12)

The image processing device according to (10),

in which in a case that a difference between a first distance betweenthe first pixel position and the center position and a second distancebetween the second pixel position and the center position is greaterthan a predetermined distance, the size acquisition unit calculates, asthe size of the insertion unit, twice the larger one of the firstdistance and the second distance.

(13)

The image processing device according to any one of (5) to (12),

in which the size acquisition unit sets the threshold on a basis of anumber-of-pixel distribution for each luminance of the endoscopic image.

(14)

The image processing device according to any one of (1) to (13),

in which the exposure control unit adjusts a parameter for controllingexposure on a basis of an output value of the image sensor in theextraction area.

(15)

The image processing device according to (14),

in which the parameter includes at least any one of an electronicshutter speed of the image sensor and a gain by which an analog signalcaptured by the image sensor is multiplied.

(16)

The image processing device according to any one of (14),

in which the parameter includes brightness of a light source.

(17)

The image processing device according to any one of (1) to (16),

in which the exposure control unit performs the exposure control basedon an output value of the image sensor in the extraction area in a caseof having determined that a shadow of the insertion unit is imaged inthe endoscopic image.

(18)

An image processing device including:

extracting, as an extraction area, an area corresponding to the size ofan insertion unit from an endoscopic image based on imaging by an imagesensor; and

performing exposure control by a processor on a basis of an output valueof the image sensor in the extraction area.

(19)

A program for causing a computer to function as an image processingdevice including:

an area extraction unit configured to extract, as an extraction area, anarea corresponding to the size of an insertion unit from an endoscopicimage based on imaging by an image sensor; and

an exposure control unit configured to perform exposure control on abasis of an output value of the image sensor in the extraction area.

(20)

An image processing system including:

a light source unit configured to emit light;

an image sensor configured to capture an endoscopic image by receivingreflected light of the light emitted by the light source unit; and

an image processing device including

-   -   an area extraction unit configured to extract, as an extraction        area, an area corresponding to the size of an insertion unit        from the endoscopic image, and    -   an exposure control unit configured to perform exposure control        on a basis of an output value of the image sensor in the        extraction area.

REFERENCE SIGNS LIST

-   1 image processing system-   100 image processing device-   110 automatic exposure control unit-   111 size acquisition unit-   112 area extraction unit-   112 exposure control unit-   113 exposure control unit-   120 image processing unit-   130 control unit-   200 insertion unit-   210 light guide-   220 illumination lens-   230 imaging unit-   231 objective lens-   232 image sensor-   240 memory-   300 light source unit-   310 white light source-   320 condenser lens-   400 display unit-   500 operation unit-   Ce center area-   Dl second scanning direction-   Dr first scanning direction-   Im1 endoscopic image-   Mp black area frame map-   N1 to N6 peripheral areas-   Rb1 black area-   Rb2 excluded area-   Rs1 observation area-   Rw2 extraction area-   Su subject-   Wl second distance-   Wr first distance

1. An image processing device comprising: an area extraction unitconfigured to extract, as an extraction area, an area corresponding tothe size of an insertion unit from an endoscopic image based on imagingby an image sensor; and an exposure control unit configured to performexposure control on a basis of an output value of the image sensor inthe extraction area.